XRCE's Participation to ImageCLEFphoto 2007

Our participation to ImageCLEFphoto07, for the first time, was motivated by assessing several transmedia similarity measures that we recently designed and developed. The object of investigation consists here in some “intermediate level” fusion approaches, where we use some principles coming from pseudo-relevance feedback and, more specifically, use transmedia pseudo-relevance feedback for enriching the mono-media representation of an object with features coming from the other media. One issue that arises when adopting such a strategy is to determine how to compute the mono-media similarity between an aggregate of objects coming from a first (pseudo-)feedback step and one single multimodal object. We propose two alternative ways of adressing this issue, that result in what we called the “transmedia document reranking” and “complementary feedback” methods respectively. This year, with a “lightly” annotated corpus of images, it appears that mono-media retrieval performance is more or less equivalent for pure image and pure text content (around 20% MAP). Using our transmedia pseudofeedback-based similarity measures allowed us to dramatically increase the performance by ∼50% (relative). Trying to model the textual “relevance concept” present in the top ranked documents issued from a first (purely visual) retrieval and combining this model with the textual part of the original query turns out to be the best strategy, being slightly superior to our transmedia document reranking method. Enriching the image annotations by extra tags extracted from an external resource (namely the Flickr database) does not offer a significant advantage in the ImageCLEF07 corpus, even if we observed an improvement using other multimedia corpora and query sets. From a cross-lingual perspective, the use of domain-specific, corpus-adapted probabilistic dictionaries seems to offer better results than the use of a broader, more general standard dictionary. With respect to the monolingual baselines, multilingual runs show a slight degradation of retrieval performance ( ∼6 to 10% relative).